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ABSTRACT 


COCKPIT  RESOURCE  MANAGEMENT:  EFFECTS  ON 
BEHAVIORAL.  lNTERy\CT10NS  ACROSS  AIRLINES 
AND  AIRCRAFT  TYPES 

by 

CATHY  COLEBROOK  CLOTHIER,  B.S..  M.A. 

SUPERVISING  PROFESSOR:  ROBERT  L.  HELMKEICH 

Due  to  the  growing  concern  over  human  errors  playing  an 
inrtruincntal  role  in  aircraft  accidents,  the  Federal  Aviation  Administration 
encouraged  airlines  to  dcve'np  Cockpit  Resource  Management  (CRM)  training 
programs  to  address  that  issue.  Baseline  data  measured  crew  interactions 
before  pilots  were  exposed  to  seminars  and  simulators.  Longitudinal  data 
described  behavioral  changes  wrought  by  the  interv'ention.  Within  airline  and 
within  Fleet  data  clearly  showed  that  crews  were  inore  effective  after  CRM 
training.  NN'ithin  airline  and  across  fieet  data  veriricd  fleet  diiYercnccs 
discovered  by  tire  First  method.  Technology  level  seemed  to  be  a  cause  of  th.c 
fleet  differcitccs  for  crews  performing  in  simulators.  Crew  si/e  affected 
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performance  both  or  the  line  and  in  the  simulator.  Finally,  the  across  airline 
and  across  fleet  data,  in  addition  to  describing  fleet  differences,  highlighted 
differences  between  the  behavioral  effectiveness  of  crews  in  different  airlines. 
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CHAFFER  ONE 


Ivan  Steiner  in  1974  wrote  "Whatever  Happened  to  the  Group  in 
Social  Psychology?"  in  this  article,  he  identified  the  roots  of  social 
psychology  as  springing  from  the  study  of  individuals  in  small  groups.  He 
also  bemoaned  the  fact  that  tire  study  of  groups  had  fallen  from  favor  at  least 
in  part  because  it  is  difficult  to  ca'^turc  the  intricacies  of  group  interactions 
in  an  experimental  setting.  Steiner  emphasized  the  need  to  study  actual 
groups  in  realistic  settings  in  order  to  capture  valid,  gcncralizablc  bch.avioral 
dynamics. 

froii'  ti'.e  study  of  groups,  social  psyciro'ogy  has  grown  to  cnccmpas.s 
and  emphasize  other  areas  of  work  such  as  the  sell  and  attribution  theory, 

Yet  our  discipline  is  a  dynamic  one  that  encourages  applied  and  pure 
research  to  play  off  of  each  other.  So  today  we  are  focusing  on  the  study  of 
small  groups  again.  Thanks  to  knowledge  gained  in  the  experimental  areas, 
applied  research  in  small  group  interactions  is  once  again  in  fas’  r.  Wiliiam 
McGuire  (1967)  extolled  the  benefits  (T  applied  and  pure  research  when  Ire 
called  for  unity  between  theory  oriented  research  and  natunal  setting 
investigations.  In  keeping  with  what  McGuire  called  'tire  best  of  both 
worlds  in  social  psychology',  my  work  in  the  aviation  field  has  involved 
applied  research  and  an  outcome  of  this  study  will  be  a  template  to  use  in 
experimental  designs  in  the  future. 
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More  specifically,  my  work  has  investigated  aircraft  accidents. 

When  airplanes  crash  we  obviously  want  to  know  why.  Was  it  due  to 
weather?  Or  was  the  accident  caused  by  catastrophic:  mechanical  failure  as  in 
the  case  of  linilcd  Flight  232  which  crashed  at  Sioux  City,  Iowa  without  its 
hydraulic  fight  controls.  Or  was  the  accide.it  due  to  human  error? 
Government  and  civilian  sources  attribute  55%  to  70%  of  all  aircraft 
accidents  to  pilot  error.  A  classic  exa.mple  of  pilot  error  occurred  in 
December  1972.  After  the  pilots  lowered  the  landing  gear  on  their  L-101 1. 
they  noticed  th.at  tlw  g'cen  nose  gear  "down  and  locked"  light  was  not  il¬ 
luminated.  Tliough  the  red  warning  light  in  tl'.c  gear  liandlc  itself  was 
extinguished,  indictiiing  throe  looked  gear,  the  crow  elected  to  iinesiigate 
furti.er.  While  attempting  to  solve  the  problem,  all  three  men  focused  on 
this  one  indicator  bulb.  No  one  noticed  Lhat  the  autopilot  had  become 
disengaged  and  that  they  w'cre  in  a  steady  descent.  They  crashed  in  the 
Florida  Fscrglades,  killing  99  people.  Did  they  have  a  m.alfunction?  Most 
dcfnitcly.  Tlic  Group  Process  malfunctioned. 

Wliat  is  the  group  process?  It  is  a  complex  intenction  between 
individuals  who  have  joined  forces  in  an  attempt  to  reach  a  goal.  To  grasp 
the  "group  process"  concept,  these  two  words  can  be  defned  individually. 
First,  a  "group"  is  more  than  simply  a  collection  of  individuals.  Fisher 
(19S0)  cites  Ju!m  K.  Bnilrart's  (1978)  fve  characteristics  of  a  group; 

1.  A  sufficiently  small  number  or  people  so  that  each  is  aware 
of  and  can  react  to  the  others. 


2.  A  mutually  interdependent  goal  in  which  tlie  success  of 

each  is  dependent  u[)on  the  success  of  others. 

3.  Sense  of  belonging. 

4.  Oral  inteiaction. 

5.  Behavior  based  on  norms  and  procedures. 

The  second  word,  "process’”,  incorporates  tlic  dynamic  relationships  of 
events  as  they  change  in  time.  These  events  affect  each  other  continuously. 
Through  process,  advancement  toward  a  goal  results.  The  group  process, 
then,  is  the  fluid,  interdependent  series  of  events  that  occur  as  the  group 
moves  toward  its  goal. 

Within  that  group  context,  two  dimensions  cxist--lhe  social 
dimension  and  the  task  dimension.  The  social  dimension  refers  to  the 
relationships  between  group  members.  Feelings  toward  others,  membership 
satisfaction,  and  group  identification  fall  under  the  social  dimension.  The 
task  tlimcnsion,  on  tlic  otlicr  hand,  refers  to  the  work  to  be  performed. 
DcHning  t!tc  Job  and  how'  it  will  be  completed  are  part  of  the  task 
dimension. 

Since  the  task  dunenslon  holds  tlie  key  to  what  takes  place  in  the 
cockpit,  a  closer  look  is  needed  at  this  dimension.  The  task  dimension  can 
be  tliough.t  of  as  a  system  containing  three  major  divisions-Inputs,  Process, 
and  Outcomes. 
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McGrath's  model  in  Figure  1  illustrates  factors  affecting  group 
performance  within  these  divisions.  Input  variables  concern  the  personal 
characteristics  members  bring  to  the  group,  tlie  composition  of  Lhe  group 
itself,  and  the  environment  in  which  the  group  is  functioning.  Process 
variables  refer  to  the  group  interaction  itself-how  individuals  coordinate 
their  actions.  Finally,  outcome  variables  de.scribe  how  well  the  group 
performed  the  Li.ik.  This  division  also  includes  changes  in  attitude  and 
cohesivencss. 

As  mentioned  above,  Uie  tliree  types  of  input  variables  concern  the 
individual,  group,  and  environmental  factors.  Within  the  individual  factor 
rests  skills,  attitudes,  and  personality  characteristics  that  each  member  brings 
to  the  interaction.  Individual  skills  determine  the  level  of  expertise  and 
resources  available  to  the  group.  Attitude  and  personality  affect  how  the 
group  members  actually  interact. 

Tlie  g'‘Oup  factor  includes  size,  prior  history,  and  structure.  Size  of 
tlic  group  affects  workload  distributions.  Prior  history  can  either  enhance  or 
inhibit  the  functioning  of  the  interaction.  Structure  refers  to  the  rights  and 
responsibilities  of  each  member  in  their  group  role. 

Environmental  factors  refer  to  the  characteristics  of  the  task  itself, 
level  of  s'rcss,  and  organizational  structure.  The  task  design  can  be  highly 
automated  or  primarily  i  anual.  Stress  f.uctuates  depending  on  environmental 
changes  such  as  severe  weather  ir.  the  aviation  Held.  Finally,  organizational 
structure  sliapcs  the  environment  and  goal  the  group  must  achieve. 
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Input  variables  in  McGrath's  model  flow  into  the  process  area. 

Within  the  process  division  lie  factors  describing  the  actual  mechanics  of 
group  interaction.  This  interaction  in  the  cockpit  context  can  be  described 
by  communication  factors  and  leadership  characteristics.  One  important 
communication  factor  is  die  preflight  briefing.  Ginnett  (1988)  studied 
airline  crews  in  their  formation  phase  and  determined  that  the  initial  briefing 
was  critical  to  the  performance  of  the  crew.  If  that  briefing,  which  was 
generally  conducted  by  die  captain,  established  an  interactive  environment, 
contained  'avc"  language,  and  defined  guidelines  for  crew  actions,  crew 
performance  was  consistently  higher.  Other  communication  markers  were 
identified  by  Foushcc  and  Manos  (1981)  in  their  analysis  of  cockpit  voice 
recordings  from  the  Ruffell  Smith  (1979)  simulation  study  of  airline  crews. 
Foushec  and  Manos  found  that  crews  who  accomplished  fewer  errors  during 
the  night  exhibited  certain  types  of  communication  styles.  These  styles  fell 
into  the  areas  of  decision-making,  assertion,  and  workload  distributions.  In 
decision-making,  a  low-error  crew  shared  the  "big  picture"  within  the  team. 
Assertion  behavior  described  crewmembers  who  stated  their  information, 
asked  questions  regarding  crew  actions,  and  advocated  a  plan  of  action.  The 
effective  crews  also  reported  excessive  workloads  when  they  occurred  and 
expressed  work  priorities  to  the  rest  of  the  crew.  Another  communication 
determinant  of  crew  coordination  was  self-critique,  liwing  Janis  (1972) 
emphasized  feedback  as  a  critical  component  in  the  decision-making  process. 
Without  sclf-crilique,  learning  and  evaluation  docs  not  take  place. 
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Leadership  characteristics  also  describe  the  group  interaction. 

Fiedler  (1967)  identified  two  basic  leadership  profiles  as  task-oriented  and 
group-oriented.  These  correspond  nicely  to  the  Sp)ence  and  Helmreich 
(1978)  instrumentality  and  exp.-es.sivity  profiles  of  behavior.  On  tlie  flight 
deck,  these  profiles  are  manifested  in  a  concern  for  tasks  and  in  the  group 
climate.  A  crew  high  in  task  concern  will  use  all  available  resources  to 
reach  a  goal.  They  will  act  decisively  when  the  situation  requires.  A  crew 
with  a  positive  group  climate  will  ensure  the  "tone"  is  relaxed  and 
supportive.  They  will  check  in  with  each  other  to  provide  support. 

In  McGrath's  model,  inputs  set  the  stage  for  the  group  process. 

What  takes  place  in  the  interaction  determines  the  outcome.  In  the  aviation 
arena,  the  outcome  hopefully  is  a  safe  and  efficient  flight.  Crews  which 
exhibit  the  commui’'  -ation  and  leadership  skills  necessary  in  the  interaction 
phase  should  produce  fewer  errors  and  be  characterized  by  cohesiveness. 

The  FAA  realizes  that  pilots  circ  technically  proficient  as  individuals. 
However,  since  55%  to  70%  of  all  aircraft  accidents  are  due  to  human 
error,  the  FAA  asked  if  pilots  car  be  made  proficient  as  a  crew.  To 
emphasize  its  concern,  the  FAA  said  in  a  circular: 

"In  recent  years  a  growing  consensus  has  occurred  in  industry 
and  government  that  training  should  emphasize  crew- 
coordination  and  die  management  of  crew  resources." 

Due  to  this  pressure,  several  airlines  instituted  their  own  Cockpit  Resource 
Management  (CRM)  programs,  CRM  is  the  "effective  utilization  of  all 
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available  resoiirces--hardwaje,  software  and  liveware-lo  achieve  safe, 
efficienl  f.ight  operations"  (i^uber,  1987).  The  question  now  is  "Are  these 
programs  producing  positive  group  behavior  changes  among  airline  crews?" 


CHAPTER  TWO 


The  first  question  that  must  be  answered  is  "How  do  we  measure  the 
effectiveness  of  behavioral  interactions?"  A  common  airline  yardstick  is  the 
accident  and  incident  rate.  But  these  are  poor  indices  for  our  purposes 
because,  thankfully,  they  are  few  and  far  between.  We  need  a  method  to 
capture  the  day-to-day  activities  of  crews  on  the  line  and  in  LOFT  (Line 
Oriented  Flight  Training).  Tlie  instrument  developed  in  response  to  this 
need  was  the  NASA/UT  Line/LOFT  Worksheet. 

Three  versions  of  the  worksheet  have  been  used.  The  first  was 
developed  in  March  1987,  It  asked  for  evaluations  of  each  crewmember 
separately.  Analysis  showed  that  evaluators  were  scoring  tlie  individual 
crewmembers  nearly  identically,  and  only  occasionally  mentioning 
significant  actions  by  a  pilot.  Therefore,  one  year  later  in  April  1988,  the 
second  version  was  put  into  use.  It  asked  for  crew,  as  opposed  to 
individual,  evaluations.  To  capture  individuality,  the  second  version  of  the 
worksheet  p.^ovided  space  for  evaluators  to  record  significant  behavior  items 
of  specific  crewmembers.  Finally,  an  even  more  refined  version  of  the 
worksheet  was  developed  in  January  1991.  The  15  behaviors  listed  on  the 
two  previous  versions  were  consolidated  into  ten  behaviors.  The  remainder 
of  the  worksheet  remained  the  same.  Examples  of  the  three  versions  of  the 
KASA/UT  Linc/LOFT  W'orksheet  are  shown  in  Appendix  A. 
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Items  to  be  scored  on  the  worksheet  came  primarily  from  two 
sources.  The  first  source  was  a  NASA  investigation  (Cooper,  White,  and 
Lauber,  1980)  in  which  60  accidents  occurring  during  1968  -  1976  were 
studied.  These  accidents,  in  which  crew  coordination  problems  had  been 
cited  as  playing  a  significant  role,  displayed  common  themes.  Among  them 
were: 

'Preoccupation  with  minor  technical  problems 
-Inadequate  leadership 

-Failure  to  delegate  tasks  and  assign  responsibilities 

-Failure  to  set  priorities 

-Inadequate  monitoring 

-Failure  to  utilize  available  data 

-Failure  to  communicate  intent  and  plans 

The  second  source  was  the  Cockpit  Management  Attitudes 
Questionnaire  (CMAQ)  (Hclmreich.  1984).  This  25-itcm  instrument  taps 
crewmember  attitudes  on  three  factors; 

1.  Communication  and  Coordination  defines  a  belief  in 
shared  information  and  teamw'ork  in  the  cockpit. 

2.  Command  Responsibility  reflects  agreement  with  the 
appropriateness  of  the  shared  responsibility  of 
crewmembers  for  the  flight. 

3.  Recognition  of  Stressor  Effects  pertains  to 
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consideration  and  possible  compensation  for  performance 
degradation  due  to  stress.  (Grcgorich,  Helmreich,  and 
Wilhelm.  1990). 

From  the  NASA  investigation  and  the  CMAQ  the  NASA/UT 
Line/LOFT  Worksheet  was  bom.  Here  1  will  discuss  only  the  latest  version 
with  its  ten  items  since  it  is  the  most  current  and  encompasses  constructs 
tapped  by  the  first  two  versions.  In  it,  three  clusters  hold  different 
behaviors.  The  first  cluster  describes  Communicalioii  Processes  and 
Decision  Behavior.  In  it  are  four  scored  behaviors: 

-Briefing  (conduct  and  quality).  The  effective  briefing  will 
be  operationally  thorough,  interesting,  and  will  address  coordination, 
planning,  and  problems.  Although  primarily  a  Captain  responsibility,  other 
crewmembers  may  add  significantly  to  planning  and  definition  of  potential 
problem  areas. 

-Inquiry/ Advocacy/ As.scrtion.  This  rating  assesses  the 
extent  to  which  crewmembers  advocate  tlie  course  of  action  they  feel  best, 
even  if  it  involves  conflict  and  disagreements  with  other. 

-Crew  Self-Critique  (decisions  and  aefinns).  This  item 
evaluates  the  extent  to  which  crewmembers  conduct  .and  participate  in  a 
debriefing,  operational  review,  and  critique  of  activities.  This  includes  the 
product,  tl’,c  process,  and  the  people  involved. 

-Communicalion/Decisioiis.  This  rating  reflects  the  extent 
to  which  free  and  open  communication  is  practiced.  It  includes  initiating 
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checklists  and  alerting  others  to  developing  problems,  Active  participation 
in  the  decision-making  process  is  encouraged  and  practiced.  Decisions  are 
clearly  communicated  and  acknowledged. 

The  second  cluster  of  behaviors  have  the  heading  Team  Building  and 
Maintenance.  Two  behaviors  are  in  this  cluster: 

-Ivcadership,  Followership,  and  Concern  for  Tasks.  This 
rating  evaluates  the  extent  to  which  appropriate  leadership  and  followership 
is  practiced.  It  also  reflects  the  extent  to  w'hich  the  crew  is  concerned  with 
the  effective  accomplishment  of  necessary  tasks. 

-Interpersonal  Relationships/Croup  Climate.  This 
cvaluaiion  captures  the  quality  of  observed  interpersonal  relationships  among 
crewmembers  and  the  overall  climate  on  the  flight  deck. 

The  third  cluster  contains  two  behaviors  that  describe  Workload 
Management  and  Situational  Awareness.  'Fhey  are: 

-Prcparation/l’lanning/Vigilance.  This  rating  indicates  the 
extent  to  wliich  crews  anticipate  contingencies  and  actions  that  may  be 
required.  Vigilant  crew  devote  appropriate  attention  to  required  tasks  and 
respond  immediately  to  new  information.  A  crew  indulging  in  casual  social 
conversation  during  periods  of  low  workload  is  not  lacking  in  vigilance  if 
fight  duties  are  being  discharged  properly  and  (he  operational  environment 
monitored. 

-\N’orkload  DIslribulion/Distraction  Avoidance.  Time  and 
workload  management  is  scored  here.  It  reflects  how  well  the  crew 
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distributed  the  tasks  to  be  accomplished.  It  also  measures  tlic  ability  of  the 
crew  to  avoid  distractions  and  to  prioritize  activities. 

The  final  two  items  are  more  global  in  nature: 

-Overall  Techuk  roficiency.  This  score  depicts  how  well 
a  crew  di.scliarges  the  technical  aspects  of  the  flight.  Demonstrated  mastery 
of  CRM  concepts  cannot  overcome  a  lack  of  proficiency.  Similarly,  highly 
proficient  technical  skills  cannot  guarantee  safe  operations  in  the  absence  of 
effective  crew  coordination. 

-Overall  Crew  Effectiveness.  This  item  is  a  composite 
judgment  of  the  crew's  performance.  It  Lakes  technical  and  CRM  bchavior.s 
into  account. 

These  Line/LOFT  Worksheet  behaviors  tap  directly  into  the  process 
division  of  McGrath's  model.  That  process  was  the  focus  of  this  three- 
pha.scd  study.  The  first  phase  took  place  within  airline  and  w'ithin  fleet. 

Two  major  domestic  airlines  allowed  their  own  check  airmen  .and 
instructors  to  gather  data  on  crews.  The.se  raters  only  observed  crews  in  the 
type  of  aircraft  in  which  the  raters  were  qualified.  Deidentified  data  from 
the  Line/LOFT  Worksheet  were  sent  to  The  University  of  Texas  at  Austin. 
The  purjrose  of  this  phase  of  the  .study  was  to  determine  if,  and  how,  the 
introduction  of  CRM  training  affected  the  interactions  of  crews  on  the  line 
and  in  LOFT.  A  secondary  purpose  was  to  investigate  the  possibility  that 
crews  perform  differently  as  a  function  of  the  type  of  aircraft  flown.  Two 
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reasons  for  potential  fleet  differences,  technology  level  and  crew  size,  were 
to  be  analyzed. 

Based  on  results  from  Lhc  first  phase  which  revealed  improvement  in 
crew  effectiveness  and  differences  in  behavior  among  various  fleets,  the 
second  phase  was  planned.  In  this  phase,  one  of  the  original  airlines 
conducted  a  one-month  audit.  Eight  evaluators  observed  crews  within  their 
own  airline,  but  across  different  aircraft  types.  The  purpose  of  this  phase 
was  to  verify  that  fleet  differences  do  indeed  exist,  arid  were  not  simply 
artifacts  of  the  method  of  taking  data  used  in  the  first  phase. 

The  third  phase  of  this  study  used  a  different  set  of  evaluators. 

Eleven  researchers  from  the  University  of  Texas  at  Austin  accomplished  the 
same  training  on  tlie  use  of  the  Line/LOFT  Worksheet  as  did  the  check 
airmen  and  instructors  in  the  two  airlines.  These  researchers  observed  crews 
in  five  airlines.  Four  of  these  were  major  domestic  airlines  a.id  one  was  a 
foreign  carrier.  This  expanded  phase  sought  to  identify  differences  between 
the  behavioral  interactions  of  crews  flying  for  different  airlines.  It  also 
targeted  feet  differences  and  the  reasons  for  those  differences. 

This  study  is  imporlam  both  for  present  and  future  implications.  By 
verifying  th.nt  crews  perform  more  effectively  after  CRM  training,  the  great 
economic  cost  of  these  progrems  is  justified  The  discovery  of  differences 
in  interaction  patterns  based  on  ai: craft  type  fuels  a  deeper  look  into  the 
reasons  for  those  differences.  By  identifying  technology  level  and  crew  size 
effects,  training  programs  can  by  designed  to  enhance  oerform''nce  despite 
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obstacles  caused  by  aircraft  type.  With  a  look  the  future,  aircraft  can  be 
designed  with  the  optimal  crew  size  and  technology  level  in  mind. 


CHAPTER  THREE 


This  thesis  addresses  data  gatliered  by  tliree  different  methcxJs.  'ITie 
first  method  took  place  within  airline  and  w-ithin  fleet.  This  means  lliat 
obsen.'crs  in  Airline  A  and  Airline  B  obtained  data  only  from  crews  within 
their  respective  airline.  Al.so,  each  observer  w'atchcd  crews  only  within  the 
type  of  aircraft  for  which  tlie  observer  was  qualified.  The  second  method 
was  within  airline  and  across  fleet.  Airline  A  accomplished  a  one  month 
audit  during  October  1990.  Eight  observ-ers  received  additional  training  to 
increase  llicir  interratcr  reliability.  Then  each  obscrv'cr  flew  with  crews  in 
several  different  types  of  aircraft.  The  final  method  of  observation,  across 
airline  and  across  fieet,  was  done  by  the  research  team  at  the  University  of 
Texas  at  Austin.  This  team  headed  by  Dr.  Robert  Helmreich,  was 
comprised  of  social  psychology  graduate  students,  research  staff,  and  retired 
Check  Airman.  Six  of  tlie  eleven  members  were  rated  pilots  whose 
experience  ranged  from  small  aircraft  to  airlines  to  military'  flying.  This 
research  team  accomplished  the  same  training  as  did  the  airline  observers. 

Observers 

l.kich  of  two  airlines  supplied  volunteer  Check  Airmen  and  Line 
Oriented  Flight  Training  instructors  to  act  as  expert  observ'ers.  These  pilots 
were  trained  m  the  basics  of  Cockpit  Resource  Management  and  then  spent 
an  additional  day  receiving  evaluator  training.  They  were  first  taught  to 
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recognize  behavioral  markers  of  effective  group  interaction.  I’sing  several 
videotaped  vignettes,  these  pilots  scored  the  behavior  of  crews  flying 
simulator  missions.  Discussion  and  comparison  after  each  vignette  increased 
interratcr  reliability. 

Airline  A  trained  707  observers  in  1989.  Airline  B  trained  a  total  of 
290  observers.  Approximately  one-third  of  these  were  trained  in  1987. 

In  both  1983  and  1989.  roughly  !00  Airline  B  observers  began  taking  data. 
Subjects 

All  pilots  in  each  airline  were  subject  to  this  anonymous  no-jeopardy 
evaluation.  Crews  were  given  no  notice  of  an  impending  CRM  evaluation 
in  cltlicr  tlie  sitnulator  or  on  the  line.  Crews  ranged  in  expertise  from 
having  absolutely  no  CRM  training  to  being  fully  trained. 

Procedures 

1  ,ach  airline  conducted  interactive  three-day  CRM  seminars  for 
groups  of  approximately  24  pilots.  These  pilots  were  divided  into  four 
groups  which  Uvcomplisheu  activities  designed  to  build  cohesiveness.  As  a 
team,  tl'.csc  groups  accomplished  tasks  centered  on  briefing  skills, 
communicauon  loop.s  conflict  resolution,  and  the  effective  use  of  all 
resource."  available  to  a  flight  crew  faced  with  dirTicull  decisions. 

SN'it'un  six  months  of  the  semi.nar  training,  pilots  at  each  airlir.e  put 
dicir  Skills  to  use  in  a  full-mission  LOFT.  These  scenarios  were  conducted  in 
high  f.dclity  full-im.tion  simulators.  Full  color  visuals  e.ihanced  t!ie  realism 
of  the  flight.  These  LOIT'  scenarios  were  designed  to  present  the  crew  with  a 


18 


problem  that  could  be  successfully  solved  if  CRM  were  employed.  After  the 
mission,  crews  reviewed  tlieir  videotaped  performance  with  the  LOFT 
instructor  and  were  encouraged  to  debrief  themselves.  Crews  erased  their 
own  tapes  after  the  feedback  session. 

Airline  A  trained  its  entire  pilot  force  in  1989.  ITiis  company  is  now 
doing  recurrent  CRM  training  for  all  crews.  Both  seminar  and  simulator 
training  is  refreshed.  Airline  B  began  tlieir  program  ii.  late  1987.  Pilots  for 
this  company  also  receive  yearly  recurrent  training. 

Response  Measure 

The  L’T/NASA  Line/LOFT  Worksheet  provides  a  five-point  Likert 
scale  ratittg  for  interactive  behaviors  which  contribute  to  two  global 
measurcs-Overall  Technical  Proficiency  and  Overall  Crew  Effectiveness. 
These  behaviors,  which  depict  effective  crew  coordination,  are  rated  from 
Poor  to  Lxccllcnt.  The  worksheet  also  provides  space  for  the  evaluator's 
comments  about  significant  behavior  or  circumstances. 


CHAPTER  FOUR 


Within  Airline  and  Within  Fleet 

My  first  question  concerned  behavior  changes  within  Airline  A. 
Airline  A  is  unique  in  that  all  pilots  were  introduced  to  CRM  concepts  in  a 
seminar  and  applied  these  concepts  in  LOFT  scenarios  within  a  12  month 
period.  I'herefore,  evaluators  had  tlie  opportunity  to  observe,  in  a  short 
period  of  time,  the  behaviors  of  pilots,  both  trained  and  untrained.  I  asked, 
"Do  crews  exhibit  changed  behavior  as  a  result  of  CRM  training?"  The 
results  were  positive  in  both  LOFT  and  line  observations.  Figures  2  through 
5  display  clianges  seen  in  LOFT  evaluations  between  crews  without 
exposure  to  CRM  and  crews  who  h-’.d  completed  both  the  seminar  and 
LOFT  training.  Over  1625  crews  comprise  the  untrained  category  and  over 
485  crews  were  in  the  fully  trained  group.  In  each  of  the  14  areas  of 
behavior  on  the  Linc/LOIT  Worksheet,  a  statistically  significant  difference 
existed  between  these  untrained  and  trained  crews.  (F(2,  3379)  =  4.37,  p 
<  .012).  In  all  cases,  fewer  crews  were  performing  at  a  substandard  level 
after  training.  At  the  highest  end  of  the  scale,  fid'y  trained  crews  generally 
earned  more  '5'  ratings  than  untrained  counterparts.  However,  the 
percentage  of  fully  trained  crews  receiving  a  rating  of  '4',  dropped  in 
comparison  to  untrained  crews. 
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A  similar  pattern  existed  in  the  line  data  from  Airline  A,  as  shown  in 
Figures  10  through  13.  In  this  case,  over  2000  untrained  crews  v/cre 
compared  with  approximately  1000  crews  tliat  had  done  some  training.  A 
partially  trained  category  was  used  because  this  sample  from  the  line  did 
not  contain  crews  in  which  every  pilot  had  received  both  the  seminar  and 
LOFF  training.  In  12  areas,  a  statistically  significant  difference  existed 
between  untrained  and  partially  trained  crews.  {F(l, 2932)  =  1.97,  p  < 

.018).  Two  areas  showed  no  significant  changes.  They  were  Concern  for 
Accompli.shmeiU  of  Tasks  at  Hand  (F(l,2932)=  1 .4,  p  =  .053),  and 
Interpersonal  Relationships/Group  Climate(F(l ,  2932)  =  . 833.  p  =  .181). 
However,  these  two  areas,  especially  Interpersonal  Rclalionships/Group 
Climate,  were  strongly  skewed  toward  the  positive  end  of  the  scale  even  in 
the  untrained  category.  Again  in  all  areas,  a  substartial  drop  in  the  number 
of  crews  rated  below  standard  w'cs  seen  among  crews  exposed  to  CRM.  On 
the  opposite  end  of  the  scale,  partially  trained  crews  earned  more  '4'  and  '5' 
ratings.  Tlicrcfore,  on  the  line,  crews  seemed  to  have  shifted  their 
performance  homogeneously  to  the  right,  whereas  in  LOFT,  crews  shifted 
upward  into  two  areas-Meets  Standards  and  Excellent.  Figures  6  through  9 
depict  performance  effects  in  LOIT',  while  Figures  14  through  17  show  the 
shift  in  pcrform.ance  of  line  crews. 


Figure  10:  Line  Ratings 


29 


OVERALL  VIGILANCE  PREPARATION  FOR  ACTIVITIES 

%  of  Ralings 


31 


CO 

O) 


cc 

cc 


0) 

c 


*NoTr.lnlno  ^  P.rll.lly  Tftinod  ^  Mo  Trolnlng  ^  Porllally  Tr.lnod 


32 


36 


37 


During  1990,  Airline  A  observers  continued  to  record  behavioral 
interactions  of  crews.  At  this  point,  every  pilot  had  received  the  seminar 
and  LOFT  training  in  1989.  Those  not  with  the  company  in  1989, 
participated  in  a  CRM  seminar  during  their  initial  training.  They  also 
accomplished  LOFT  scenarios  during  the  year.  A  valuable  comparison  of 
behaviors  in  Airline  A  across  time  is  shown  in  Figure  18.  Crews  improved 
the  most  in  the  Briefing  and  Crew  Self-Critique  categories,  gaining 
approximately  one-1  alf  a  standard  deviation.  Least  improvement  was  seen 
in  the  categories  of  Task  Concern  and  Technical  Proficiency. 

Having  seen  how  crews  behaved  in  the  cockpit  before  and  after 
indoctrination  into  the  concepts  of  CRM,  the  next  question  was,  "What 
behaviors  most  often  trigger  excellent  ratings  and  correspondingly,  what 
behaviors  most  often  trigger  poor  ratings?"  Armed  whth  6804  valid 
observ'ations  from  1989,  I  found  that  Airline  A  evaluators  gave  excellent 
scores  most  often  in  the  areas  of  Interpersonal  Relationships/Group  Climate, 
Briefing,  and  Task  Concern.  On  the  other  hand,  evaluators  gave  poor 
scores  most  often  in  the  areas  of  Asseriion/ Advocacy,  Distractions 
Avoided  or  Prioritized,  and  Briefing.  Figures  19  and  20  depict  the  be¬ 
haviors  that  most  often  trigger  excellent  or  poor  ratings. 


Figure  20:  What  Triggers 
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In  1990,  Airline  A  observers  said  that  the  triggers  of  excellent 
behavior  were  Interpersonal  Rclatior.ships/Group  Climate,  Briefing,  and 
Vigilance.  On  the  opposite  side  of  the  spectrum,  the  most  often  cited  poor 
behaviors  were  Assertion/ Advocacy,  Distractions  Avoided  or  Prioritized, 
and  Communication.  This  airline  showed  much  overlap  in  its  key  excellent 
and  poor  behaviors.  A  point  worth  noting  is  that  Briefings  were  no  longer 
drawinp  a  substantial  number  of  poor  ratings. 

The  Airline  A  data  v/cre  unique  in  that  all  crewmembers  were 
exposed  to  and  practiced  CRM  concepts  in  the  simulator  during  a  one  year 
period.  On  the  other  hand.  Airline  B  trained  its  crews  as  they  became  due 
for  their  annual  training:  however.  Airline  B  was  unique  in  that  it  had  a 
history  of  emphasizing  crew  concept  flying. 

Again  I  asked  the  same  question,  "Do  crews  exhibit  changed 
behavior  due  to  CRM  training?"  The  answer  to  this  question  was  very 
muddled  for  Airline  B.  At  first  glance  the  raters  were  reporting  that  crews 
with  no  training  in  1987  were  performing  at  the  same  level  as  completely 
trained  crews  in  1989.  However,  an  initial  analysis  of  the  data  in  mid- 1988 
had  found  improvements  in  crew  behavior.  Based  on  this  lead,  1  examined 
only  the  1987  data.  Indeed,  crews  that  had  been  exposed  to  the  formal 
CRM  program  were  outperforming  their  untrained  counterparts  in  the  areas 
of  Inquiry,  Decisions  Communicated,  Task  Concern,  Interpersonal 
Relationships/Group  Climate,  Preparation,  ana  Overall  Crew  Effectiveness. 
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(F(l,174)=2.43.  p  <  .042).  Upon  scnitiny  of  the  1988  data,  I  found  that 
the  airline  observers  snw  no  difference  between  trained  and  untrained  crews. 
That  effect  held  for  the  1989  data  as  well. 

Since  an  anchoring  effect  could  have  caused  this  improvement  and 
then  perceived  degradation  of  performance,  the  demographics  of  the  raters 
themselves  was  examined.  Airline  B  had  a  core  of  126  evaluators 
accomplishing  the  Line/LOFT  Worksheet.  Of  those,  35%  began  sending 
data  in  1988  and  34%  began  recording  data  in  1989.  The  instruction 
concerning  Worksheet  completion,  howc.er,  took  place  in  1987.  After 
conferring  with  the  airline  managers  responsible  for  tlieir  program,  a  very 
plausible  explanation  for  tlie  results  emerged.  New  observers  were  simply 
scoring  behavioral  interactions  based  on  their  own  intuition.  However  valid 
that  intuition  may  have  been,  it  was  skewed  by  the  organizational  influences 
that  had  already  occurred  in  Airline  B.  Since  documented  performance 
improvements  were  shown  in  1987,  crews  in  Airline  B  were  interacting 
more  cftlcicntly.  When  new  raters  came  on  line  in  1988  and  1989,  they 
w'ere  observing  fully  CRM-trained  crew's.  These  raters,  unslandardizcd  in 
using  the  five-point  rating  scale,  possibly  used  the  midpoint,  3,  to  imply 
"average".  However,  the  average  Airline  B  crew  was  actually  performing  at 
a  higher  level  than  in  1987.  This  finding  highlighted  the  importance  of 
standardized  observ'ers.  Airline  B,  and  also  Airline  A,  heeded  this  warning 
and  conducted  additional  training  sessions  for  their  raters. 
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Though  an  anchoring  problem  existed,  it  did  not  affect  the  triggers  of 
excellent  and  poor  behavior  of  Airline  B  crews.  Observers  gave  highest 
scores  most  often  in  the  areas  of  Task  Concern,  Interpersonal 
Relationships/Group  Climate,  and  Vigilance.  Poor  scores  were  most  often 
given  for  Crew  Self-Critique,  Inquiry,  and  Assertion.  See  Figures  21  and 
22. 


Figure  21:  What  Triggers 
Excellent  Ratings  in  Airline  B? 


Figure  22:  What  Triggers 
Poor  Ratings  in  Airline  B? 
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Having  shown  that  CRM  train-ng  did  indeed  improve  the 
performance  of  crews  in  two  different  airlines,  the  rich  data  set  was  ripe  for 
a  deeper  look.  The  most  obvious  area  to  explore  was  the  fleet  differences. 
Did  crews  behave  differently  in  the  different  aircraft  types?  The  Airline  A 
data  provided  an  answer.  (Airline  B  data  was  not  examined  bcvau:.e  cf  its 
anchoring  problem.) 

A  MANOVA  done  on  6129  cases  from  1989  yielded  a  significant 
difference  betw'een  eight  fleets,  F=2.49,  p  =  .000.  Specific  behavioral 
ratings  driving  the  difference,  using  alpha  =  .05/14  =  .0038,  were  all 
categories  except  Decisions  Communicated,  Vigilance,  and  Distractions 
Avoided.  Figure  23  depicts  the  means  by  fleet  for  Technical  rroficiency  and 
Overall  Crew  Effectiveness. 

For  the  data  gathered  in  1990,  a  MANOVA  on  3756  cases  replicated 
the  significant  difference  found  in  the  "’•evious  year,  F  =  2.034,  p  =  .000. 
Significant  behavioral  ratings  causing  the  difference,  again  using  alpha  = 
.0038,  were  all  categories  except  Advocacy,  Decisions  Communicated, 
Interpersonal  Skills/Group  Climate,  and  Technical  Proficiency.  Figure  24 
shows  the  means  by  fleet  for  Technical  Proficiency  and  Overall  Crew 
Effectiveness. 
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Knowing  that  crews  were  interacting  differently  dej)ending  on  which 
aircraft  type  they  were  flying,  the  next  step  was  to  find  reasons  for  that 
difference.  Two  possibilities  were  advanced  versus  standard  technology  and 
2-person  versus  3-person  crews.  Data  from  1989  observations  in  Airline  A 
were  subjected  to  a  MANOVA  to  compare  the  two  technology  levels.  This 
analysis  was  separated  into  Line  and  LOFT  observations,  since  these 
contrasting  types  of  observations  were  significantly  different. 

The  hypothesis  that  behavioral  characteristics  of  crews  in  advanced 
technology  cockpits  differ  from  the  behavioral  characteristics  of  crews  in 
standard  instrument  cockpits  was  supported  by  the  data.  A  significant 
difference  (MANOVA  F  =  3.27,  p  =  .000)  existed  between  2761  advanced 
cockpit  crews  and  standard  cockpit  crews  on  the  line.  Areas  dri\’ing  that 
difference  were  Briefings  and  Task  Concern  with  standard  technology  crews 
outperforming  advanced  technology  crews.  See  Table  1  for  performance 
means. 


Table  1 


Mean  Ratings  of  Line  Crews  in  1989 


ADVANCED 

TECHNOLOGY 

STANDARD 

BRIEFING 

3.19 

3.28 

COMMUNICATION 

3.20 

3.27 

INQUIRY 

3.16 

3.19 

ASSERTION/ ADVOCACY 

3.12 

3.14 

DECISIONS  COMMUNICATED 

3.21 

3.27 

SELF-CRITI  ,'UE 

3.09 

3.06 

TASK  CONCERN 

3.27 

3.37 

GROUP  CLIMATE 

3.39 

3.43 

VIGILANCE 

3.23 

3.27 

PREPARATION/PLANNING 

3.22 

3.22 

DISTRACTIONS  AVOIDED 

3.06 

3.07 

WORKLOAD  DISTRIBUTED 

3.12 

3.15 

1  ..CHNICAL  PROFICIENCY 

3.25 

3.33 

CREW  EFFECTIVENESS 

3.29 

3.33 

Likewise  in  LOFT,  3325  advanced  and  standard  instrument  cockpit 
crews  differed  significantly,  (MANOVA  F=2.73,  p  =  .000).  No  areas  stood 
out  as  causing  the  difference,  however  the  14  categories  of  behavioral 
characteristics  show  a  trend  in  the  same  direction.  In  the  LOFT  observations, 
advanced  technology  crews  outperformed  the  standard  technology  crews. 
Table  2  shows  these  results. 


Table  2 

Mean  Raiings  of  LOFT  Crews  in  1989 


ADVANCED 

TECHNOLOGY 

STANDARD 

BRIEFING 

3.41 

3.37 

COMMUNICATION 

3.32 

3.27 

INQUIRY 

3.32 

3.25 

ASSERTION/ADVOCACY 

3.27 

3.20 

DECISIONS  COMMUNICATED 

3.37 

3.29 

SELF-CRITIQUE 

3.24 

3.24 

TASK  CONCERN 

3.37 

3.37 

GROUP  CLIMATE 

3.48 

3.43 

VIGILANCE 

3.35 

3.31 

PREPARATION/PLANNING 

3.34 

3.32 

DISTRACTIONS  AVOIDED 

3.09 

3.15 

WORKLOAD  DISTRIBUTED 

3.21 

3.21 

TECHNICAL  PROFICIENCY 

3.35 

3.36 

CREW  EFFECTIVENESS 

3.41 

3.34 

In  1990,  a  different  pattern  resulted.  The  M.^NOVA  done  on  the 
line  data  was  not  significant.  No  statistical  difference  was  seen  by  observers 
of  958  advanced  and  standard  technology  crews,  F  =  .86,  p  =  .603. 
However,  a  significant  difference  was  evident  in  LOFT,  F  =  2.60,  p  = 

.001 ,  for  a  sample  size  of  266  i .  All  behavioral  interaction  categories, 
except  Workload  Distributed,  were  driving  the  difference.  Table  3  shows 
the  mean  ratings  of  LOFT  crews  in  1990. 


Tables 

Mean  Ratings  of  LOFT  Crews  in  1990 
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ADVANCED 

TgCHi^OLQQY 

STANDARD 

BRIEFING 

3.70 

3.56 

COMMUNICATION 

3.59 

3.46 

INQUIRY 

3.59 

3.41 

ASSERTION/ADVOCACY 

3.47 

3.36 

DECISIONS  COMMUNICATED 

3.54 

3.44 

SELF-CRITIQUE 

3.44 

3.33 

TASK  CONCERI>I 

3.61 

3.50 

GROUP  CLIMATE 

3.71 

3.60 

VIGILANCE 

3.58 

3.45 

PREPARATION/PLANNING 

3.57 

3.47 

DISTRACTIONS  AVOIDED 

3.37 

3.29 

WORKLOAD  DISTRIBUTED 

3.46 

3.37 

TECHNICAL  PROFICIENCY 

3.60 

3.49 

CREW  EFFECTIVENESS 

3.64 

3.52 

Figures  25  and  26  depict  the  critical  behaviors  for  crews  flying 
LOFT  scenarios  in  1990.  Figure  25  contrasts  the  behaviors  for  which  crews 
earn  highest  scores  in  advanced  and  standard  technology  aircraft.  Figure  26 
contrasts  the  lowest  scored  behaviors  in  the  two  types  of  aircraft. 
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Besides  the  advanced  versus  standard  technology  explanation,  fleet 
differences  could  exist  due  to  the  2-peison  versus  3-person  interactions. 
Again,  this  question  was  analyzed  separately  for  line  and  LOFT 
observations.  In  1989  line  observations,  a  significant  difference  existed 
between  2775  2-person  and  3-person  crews,  (MANOVA  F  =  4.47,  p  = 
.000).  Areas  driving  that  difference  were  Crew  Self-Critique  and 
Distractions  Avoided.  See  Table  4. 


Table  4 

Mean  Ratings  of  Line  Crews  in  1989 


BRIEFING 

COMMUNICATION 

INQUIRY 

ASSERTION/ADVOCACY 

DECISIONS  COMMUNICATED 

SELF-CRiriQUE 

TASK  CONCERN 

GROUP  CLIMATE 

VIGILANCE 

PREPARATION/PLANNING 
DISTRACTIONS  AVOIDED 
WORKLOAD  DISTRIBUTED 
TECHNICAL  PROFICIENCY 
CREW  EFFECTIVENESS 


2-.P£P.SQN 

3-PERSON 

3.23 

3.30 

3.26 

3.26 

3.20 

3.15 

3.13 

3.15 

3.25 

3.26 

3.09 

3.02 

3..^5 

3.32 

3.44 

3.38 

3.27 

3.23 

3.22 

3.20 

3.10 

3.01 

3.16 

3.10 

3.33 

3.27 

3.34 

3.28 

Significant  differences  also  were  evident  in  3335  LOFT  observations, 


(MANOVA  F  =  1.96,  p  =  .017).  In  this  case,  Briefings  and  Crew  Self- 
Critique  accented  the  2-person  versus  3-person  distinction.  Table  5  shows 
LOFT  means. 

Table  5 

Means  Ratings  for  LOFT  Crews  in  1989 


2-PERSON 

3-PERSON 

BRIEFING 

3.41 

3.34 

COMMUNICATION 

3.30 

3.26 

INQUIRY 

3.29 

3,24 

ASSERTION/ADVOCACY 

3.22 

3.19 

DECISIONS  COMMUNICATED 

3.32 

3.28 

SELF-CRITIQUE 

3.28 

3.21 

TASK  CONCERN 

3.40 

3.34 

GROUP  CLIMATE 

3.46 

3.42 

VIGILANCE 

3.32 

3.31 

PREPARATION/PLANNING 

3.34 

3.30 

DISTRACTIONS  AVOIDED 

3.14 

3.14 

WORKLOAD  DISTRIBUTED 

3.22 

3.21 

TECHNICAL  PROFICIENCY 

3.38 

3.34 

CREW  EFFECTIVENESS 

3.38 

3.33 

In  1990,  Airline  A  crews  replicated  their  1989  behavior.  Again  on 
the  Line,  a  MANOVA  showed  that  each  type  of  crew  behaved  significantly 
different  from  the  other,  F  =  2.94,  p  =  .000.  Sample  size  was  958.  Crew 
Self-Critique  and  Workload  Distributed  were  the  critical  behaviors.  Mean 
Ratings  on  the  line  in  1990  are  shown  in  Table  6. 
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Table  6 

Mean  Raiings  of  Line  Crews  in  1990 


2-PERSQM 

3-PERSON 

BRIEFING 

3.41 

3.36 

COMMUNICATION 

3.38 

3.32 

INQUIRY 

3.33 

3.23 

ASSERTION/ADVOCACY 

3.28 

3.24 

DECISIONS  COMMUNICATED 

3.41 

3.35 

SELF-CRITIQUE 

3.21 

3.05 

TASK  CONCERN 

3.42 

3.37 

GROUP  CLIMATE 

3.52 

3.54 

VIGILANCE 

3.39 

3.32 

PREPARATION/PLANNING 

3.32 

3.29 

DISTRACTIONS  AVOIDED 

3.22 

3.13 

WORKLOAD  DISTRIBUTED 

3.32 

3.18 

TECHNICAL  PROFICIENCY 

3.37 

3.28 

CREW  EFFECTIVENESS 

3.42 

3.36 

Also  in  LOFT,  a  MANOVA  revealed  a  significant  crew  size 
difference,  F  =  5.63,  p  =  .000,  N  =  2661.  AH  scored  behaviors  except 
Advocacy  and  Distractions  Avoided  were  significant,  p  =  .000.  Table  7 
depicts  LOFT  behavior  means. 


Table  7 


Mean  Ratings  for  LOFT  Crews  in  1990 


2-PERSON 

3-PERSON 

BRIEFING 

3.68 

3.52 

COMMUNICATION 

3.58 

3.43 

INQUIRY 

3.53 

3.39 

ASSERTION/ADVOCACY 

3.41 

3.37 

DECISIONS  COMMUNICATED 

3.53 

3.42 

SELF-CRITIQUE 

3.45 

3  79 

TASK  CONCERN 

3.61 

3.46 

GROUP  climate 

3.71 

3.57 

VIGILANCE 

3.56 

3.43 

PREPARATION/PLANNING 

3.55 

3.45 

DISTRACTIONS  AVOIDED 

3.35 

3.29 

WORKLOAD  DISTRIBUTED 

3.47 

3.33 

TECHNICAL  PROFICIENCY 

3.60 

3.46 

CREW  EFFECTIVENESS 

3.63 

3.49 

Since  significant  differences  within  cockpits  of  different  technology 
levels  and  cockpits  of  different  crew  sizes  were  found,  I  investigated  the 
possibility  of  a  confound  between  the  two  types.  A  confound  could  have 
existed  since  the  advanced  technology  cockpits  were  flown  by  2-person 
crews.  However,  MANOVA  analyses  showed  that  this  was  not  the  case.  By 
analyzing  only  2-person  crews,  I  found  a  significant  difference  in  behavioral 
performance  between  advanced  and  standard  cockpit  crews,  replicating  in 
the  same  directions  as  the  1989  and  1990  data  reported  earlier.  See  Table  8. 
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Table  8 

Technology  Level  Differences  Within  2-Person  Crews 

1282 

Line  F  =  3.04  p  =  .000  N  =  1870 

LOFT  F  =  2.79  p  =  .000  N  -  1691 

mo 

Line  F  =  0.79  p  =  .676  N  =  613 

LOFT  F=1.8r,  p  =  .025  N  =  1034 

Also,  by  analyzing  only  standard  technology  crews,  a  significant 
difference  due  to  crew  size  was  again  rep’icated.  In  buih  1989  and  1990 
data  sets,  2-pcrson  crews  tended  to  outiierform  their  3-person  counterparts. 
See  -^'ablc  9. 


Crn^  Size  Differences  Within  Standard  Technology  Crews 


I9S2 


Line 

F=4.41 

p  =  .000 

N  =  2080 

LOFT 

F=2.12 

p  =  .009 

N  =  2608 

1990 

Line 

F  =  2.86 

p  =  .000 

N  =  648 

LOFT 

F  =  5.07 

p  =  .000 

N  -  2101 

Within  Airl 

me  and  Across  Fleet 

The  first  data  gathering  methcxl,  within  airline  and  within  fleet, 
yielded  impetant  information.  It  showed  that  crews  in  two  different  airlines 
improved  their  Tfectiveness  in  the  cockpit  as  a  result  of  CRM  training.  It 
also  highlighted  significant  differences  between  crews  Tying  different 
aircraft  types.  To  determine  if  these  differences  were  subculture  artifacts  or 
actual  disparities,  Airline  A  accomplished  a  one-month  audit  of  its  crews. 
Eight  obsen'crs  received  additional  training  on  the  completion  of  the 
Linc/LOET  Worksheet  to  insure  that  all  of  tliem  were  thoroughly  anchored 
in  the  five-point  scale.  During  October  1990,  these  eight  raters  did  172  line 
observations  within  their  own  airline,  but  acro.ss  different  aircraft  types. 


Verifying  the  earlier  find  that  crews  were  interacting  more 
effectively,  these  eight  instructors  recorded  higher  ratings  in  all  categories 
tlian  had  been  seen  during  tlie  pre-CRV;  audit  done  in  1989.  Figure  27 
compares  the  scores  obtained  during  each  audit.  Behaviont!  dimensions 
which  improved  the  most  were  Task  Concern,  Inquiry,  and  Briefing,  l^ist 
improvement  was  seen  h-  Workload  Distributed,  as  depicted  in  Figure  28. 

Airline  A  observers  most  often  rated  Interperson.  Skills/Grov'p 
Climate,  Briefing,  and  Task  Concern  as  being  above  standards.  They  saw 
Communication,  Crew  Self-Critique,  and  Decisions  Communicated  as  most 
often  below  standards.  Figure  29  shows  their  findings. 


Figure  27:  Characteristics  of 
Behavioral  Ratings 
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A  major  interest  in  this  audit  was  the  fleet  differences  found  in  the 
within  airline  and  within  fleet  study.  Though  the  sample  size  was 
inadequate  for  a  MANOVA  analysis,  by  an  ANOVA  it  revealed  two 
differences.  One  was  in  the  category  of  Task  Concern  (F(6,  151)  =3.21,  p 
=  .005).  The  other  was  in  Overall  Crew  Effectiveness  (F(6,  154)  =9.27,  p 
=  .002).  An  interesting  finding  was  that  crews  flying  different  aircraft 
were  not  significantly  different  on  the  Technical  Proficiency  dimension. 

An  ANOVA  comparing  advanced  technology  to  standard  cockpit 
crews  was  significant  only  in  tlic  area  of  Task  Concern  (F(l,156)  =  7.49,  p 
=  .007).  In  a  reversal  of  the  within  airline  and  within  fleet  1989  data,  the 
advanced  technology  crews  had  higher  means  for  Task  Concern,  Technical 
Proficiency  and  Overall  Crew  Effectiveness.  Table  10  depicts  these  means 

Table  10 

Mean  Ratings  of  Line  Crews  During  the  Audit 


ADVANCED 

TECHNOLOGY  STANDARD 

TASK  CONCERN  3.79  3.52 

TECHNICAL  PROFICIENCY  3.56  3.40 

CREW  EFFECTIVENESS  3.60  3.44 


The  2-person  versus  3-person  comparison  was  significant  in  three 
areas.  Decisions  Communicated,  (F(l,146)  =  4.32,  p  =  .039),  and  Task 
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Concern,  (F(1,156)  =  7.41,  p  =  .007)  were  significantly  different.  Also, 
significant  was  the  Overall  Crew  Effectiveness  category  {F(l,159)  =  4.04, 
p  =  .046).  Again,  Technical  Proficiency  differences  were  not  i^ignificant. 
Two-person  crews  continued  to  outperform  their  3-person  counterparts.  See 
Table  11. 


Table  1 1 

Mean  Ratings  of  Line  Crews  During  the  Audit 


2 -PERSON 

3 PER SON 

DECISIONS  COMMUNICATED 

3.49 

3.23 

TASK  CONCERN 

3.75 

3,45 

TECHNICAL  PROFICIENCY 

3.52 

3,42 

CREW  EFFECTIVENESS 

3.60 

3.35 

Though  the  audit  sample  size  was  small,  it  yielded  rich  descriptive 
statistics.  The  following  four  figures  show  how  a  particular  rater  saw  fleet 
variations  in  two  global  categories.  Technical  Proficiency  and  Overall  Crew 
Effectiveness.  The  first  three  bars  (two  for  P.ater  ^7730)  depict  that  rater’s 
means  for  particular  aircraft  types.  The  last  set  of  bars  in  e.ach  category 
depict  the  overall  mean  derived  from  all  172  observations  for  particular 
pirrraft  iyne«.  These  four  raters  were  chosen  simply  because  they  had  done 
five  or  more  obsen  ations  in  the  aircrau  depicted  in  their  graph.  Figures  30 
through  33  follow. 


Figure  30:  Rater  #1288 
Fleet  Differences 
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Figure  31:  Rater  #3145 
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Figure  32:  Rater  #6670 
Fleet  Differences 


Figure  33:  Rater  #6670 


Another  rich  source  of  information  was  the  comments  these 
evaluators  recorded  on  their  worksheets.  The  comments  highlight  both 


effective  and  ineffective  interaction  styles  taking  place  in  tlie  different  types 
of  airev- ee  Appendix  B  for  these  descriptions. 

Across  Airlines  and  Across  Fleets 

The  final  method  of  viewing  behavioral  interactions  is  much  broader 
in  scope.  Having  confirmed  behavioral  improvements  and  fleet  differences 
in  effectiveness  by  the  previous  two  methods,  it  was  time  to  obtain  an 
unbiased  view  across  several  airlines.  The  small  group  of  NASA/UT 
researchers  mentioned  earlier  was  trained  in  the  use  of  tlie  Line/LOFT 
Worksheet  and  then  began  doing  observations  across  five  different  airlines  in 
December  1990.  As  of  1  April,  1991,  the  group  had  observed  115  crews  on 
the  line  and  in  LOFT.  Since  only  six  observations  were  done  on  LOFT 
crews,  these  have  been  excluded  from  the  analysis. 

One  of  my  first  interests  was  to  explore  the  fleet  differences  found  by 
the  previous  two  methods.  Even  with  this  small  sample  size,  fleet 
differences  were  found  in  the  areas  of  Communicalion/Decisions  (F(13,80) 

=  2.89,  p  =  .002)  and  Prcparation/Planning/Vigilance  (F(12,83)  =  2.00,  p 
=  .  035).  Figure  34  depicts  means  in  these  categories  from  aircraft  in 
which  more  than  three  observations  were  accomplished. 
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Since  fleet  differences  existed  in  this  set  of  data  gathered  across 
airlines  and  across  fleets,  the  next  .ogical  question  was  to  explore  the 
pcissibility  that  technology  level  differences  w'cre  driving  die  disparities. 
Unfortunately,  only  6  of  die  ratings  were  done  on  crews  flying  advanced 
technology  aircraft.  Tliis  ana. sis  must  wait  until  the  data  set  matures; 
however,  on  a  purely  descriptive  level,  tlic  advanced  technology  crews 
outpierformcd  their  counterparts  in  nine  of  tcii  categories.  (Standard 
technology  crews  earned  higher  ratings  only  in  the  Decisions  Commun'cated 
category.)  Whether  these  are  signifleant  differences  is  something  a  larger 
sampio  \'.  ill  reveal. 

llic  other  reason  behind  iiiesc  nect  d;*''e.'’ences  could  be  the  crew 


si7.e.  An  ANOVA  was  significant  only  in  die  category  of 
Communication/Dccisions  (F(1 ,92)  =  1.54,  p  =  .049).  Within  this 
category,  2-person  cicv.'s  earned  a  mean  of  3.29  and  3-pcrson  crews  e<uiicd 


a  mean  o' 


S  S' 


Note  though  that  out  oi  ten  categories,  2 -person  crews  had 


higher  means  than  did  a-oerson  crews  in  six  categories. 

An  op[xiru;nuy  i.r  tnis  portion  of  ih.e  study  existed  to  objectively 
compare  crew  pcrfoi  inance  across  airlines.  I'hough  the  amline  industry 
prides  itself  on  its  standardization,  significant  diff  rcncCj  do  exist.  Fable  12 
shosvs  sU'.ere  tliesc  differences  are  significaiit.  Figure  .d'  tlien  shosxs  tlie 
mean  r.iimgs  ■'!  these  airlines  in  u.ese  pa-licular  categories. 


Table  12 


Categories  Showing  Significant  Differences  Between  Airlines 


Category 

df 

F 

P 

Briefing 

4.  90 

6.86 

.000 

Inquiry/ Advoc/ Assertion 

4,  90 

3.80 

.007 

Crew  Self-Critique 

4.  90 

5.94 

.000 

Technical  Proficiency 

4,  92 

1.56 

.010 

En'Ci'i  in  Its  early  stages,  this  small  sample  already  confirms  r.eet 
differences,  hints  at  technology  and  crew  size  differences,  and  for  the  first 
‘  c  illustrates  that  airlines  do  indeed  differ  in  how  Uieir  crews  perform.  As 
it  matures,  this  methexi  of  data  gathering  will  enhance  our  view  of  crew 
interactions. 


Figure  35:  Means  for  Categories  In 
Which  Airline  Differences  Exist 
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Airline  D  B  Airline  E 


CHAPTER  FIVE 


1 


Based  on  data  from  these  two  airlines,  Cockpit  Resource 
Management  training  has  improved  the  behavioral  interaction  effectiveness 
of  crews  on  the  line  and  in  LOFT.  Airline  A  experienced  the  clearest 
improvement.  This  airline  carefully  trained  its  'lata  takers  and  implemented 
an  aggressive  program  for  all  of  its  pilots  in  ?  2-month  period.  Airline  B, 
while  also  showing  improvement,  fell  prey  to  the  hazr  ds  of  untrained, 
unanchorcd  observers.  Airline  B  crews  exhibited  significant  improvement  in 
their  interactions  during  1987.  However,  by  1989,  approximately  69%  of 
the  obsers'ers  recording  data  had  ne^'cr  been  anchored  in  the  five-point  scale. 
By  that  point,  they  were  watching  only  fully  bained  crews  and  their  ratings 
muddied  the  clear  data  obtained  earlier.  Therefore,  the  importance  of 
anchored  observers  was  highlighted  in  that  study. 

An  interesting  finding  concerning  fleet  differences  trccurred.  Though 
the  airline  industry  prides  itself  on  being  highly  standardized,  crews  from 
different  fleets  obviously  interact  in  different  ways.  The  data  from  the 
within  airline  and  within  fleet  method  strongly  supported  the  hypothesis  that 
significant  differences  in  behaviors  exist  between  fleets.  One  of  the  reasons 
for  that  difference  can  be  attributed  to  the  advanced  technology  cockpit 
versus  standard  cockpit  configuration.  Airline  A  crew's  in  1989,  exhibited 
significant  differences  based  on  their  configuration  with  advanced  cockpit 
crews  performing  at  a  higher  level  in  LOFT  ,  but  with  standard  cockpit 
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crews  outperforming  their  counterparts  on  the  line,  ’ilie  following  year, 
when  all  pilots  had  completed  CRM  training,  no  significant  difference  was 
noted  between  the  two  technologies  on  the  line,  but  advanced  technology 
crews  continued  to  significantly  dominant  in  LOFT. 

The  within  airline  and  across  fleet  line  audit  done  by  Airline  A 
replies  ted  the  findings  from  the  within  airline  and  within  fleet  method. 

Fleet  differences  did  exist  on  the  line,  however  the  technology  level  was  not 
a  significant  cause,  though  advanced  technology  crews  still  tended  to  earn 
higher  ratings.  Perhaps  the  LOFT  environment,  which  emphasizes 
abnormal  situations,  allows  advanced  technology  cockpit  crews  to  put  that 
technology  to  use  in  flying  the  aircraft.  Then  crews  are  allowed  more  time 
to  gather  information  about  the  problem,  discuss  options,  and  decide  on  a 
course  of  action,  thereby  fully  using  each  other's  knowledge.  Standard 
technology  crews  do  not  have  the  option  of  telling  the  aircraft  to  fly  to  a 
specific  point  in  space.  They  must  be  more  actively  involved  in  flying, 
which  decreases  the  amount  of  lime  and  resources  they  can  contribute  to 
solving  a  problem. 

Another  reason  for  the  fleet  differences  seems  to  be  crew  size. 

Again  using  Airline  A  data,  significant  differences  were  found  between  2- 
person  and  3-pcrson  crews.  Two-person  crews  consistently  outperformed  3- 
person  crews  in  both  line  and  LOFT  operations  during  1989  and  1990. 
Two-person  crews  also  earned  higher  ratings  during  the  within  airline  and 
across  ficct  study.  While  the  third  person  is  an  extra  set  of  eyes,  that  extra 
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communication  node  seems  to  detract  more  than  aid  behavioral  operations. 
Keeping  in  mind  the  early  studies  of  groups,  it  is  interesting  to  note  that 
Simmel  (1950)  was  impressed  with  the  cohesiveness  of  the  dyad.  He 
proposed  that  a  triad  would  be  plagued  by  the  formation  of  coalitions  and 
imprecise  communication,  while  the  dyad  would  be  more  effective  in 
reaching  its  goals. 

Other  reasons  for  fleet  differences  certainly  exist.  The  positive 
ratings  of  crews  flying  the  B-737  and  the  MD-88  could  be  due  to  the 
younger  crews  typically  flying  those  aircraft.  These  pilots  could  be  more 
accepting  of  CRM  concepts  because  they  consider  tlie  training  just  one 
among  many  steps  needed  to  make  them  safe  pilots.  Also,  younger  pilots 
may  have  'grown  up'  in  a  less  authoritarian  flying  environment  than  did  the 
older  captains  who  settle  comfortably  into  flying  the  B-727.  Having  a 
backgroup't  of  cooperation  would  encourage  younger  pilots  to  be  more 
ir  volvcd  crewmembers.  Unfortunately,  the  three  data  sets  used  in  this  study 
c:.nnot  address  these  speculations,  but  future  investigations  should  examine 
tl  cse  and  may  discover  other  reasons  for  fleet  differences. 

This  study  also  discovered  significant  differences  between  crews 
flying  for  different  airlines.  Though  the  airline  industry  is  supposedly 
highly  standardized,  evidence  shows  that  different  airline  crews  clearly  fall 
on  a  continuum  of  effective  performance. 


In  answer  to  the  FAA's  question,  "Does  CRM  training  effect  the 
performance  uf  airline  crews?",  the  answer  is  a  definite  "Yes".  Since  the 
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inception  of  CRM  training  programs  in  two  airlines,  crew  performance  has 
improved.  This  study  has  also  highlighted  significant  differences  between 
fleets  that  can  be  attributed  in  part  to  technology  level  and  crew  size.  Two- 
person  crews  using  advanced  technology  are  the  most  effective  in  dealing 
with  the  abnormal  situations  presented  in  LOFT.  On  the  line,  crew  size  is  a 
factor  in  the  fleet  differences  with  2'person  crews  once  again  performing  at 
a  higher  level.  While  technology  level  was  not  a  significant  factor  in  fleet 
differences  in  day-to-day  line  operations,  advanced  technology  cockpit  crews 
tend  to  earn  higher  scores  for  their  crew  effectivenes'. 
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NASA/UT  _  LiNEyLOFT  WORKSHEET 


V.  CREW  ATMOSPHERE  AND  COORDINATION 
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NASA/UT  LINE/LOS  CHECKLIST 


I.  FLIGHT  AND  EQUIPMENT  INFORMATION 

Airline  Date  (Mo.Yr)^ _  RATER  ID. 


i-orr  Lj 

Sccnino  ID 


LOE 


SPO'' 


UNE  Check  is  t  UJ  j__J  Un*  Obi<rv»rort 
Ptlot  CA  !  ^  FO 


routino 


AJC  Typ«  4  S4n«t  I 


I 


OsmoflraDhics 

Compistsd  Initial  CRM  (Y,  N.  or  ?1 
Numbar  of  CRM  Hacurrant  Saasions 
Number  of  LOS/IOFT  Sasiions 
Crevwmembflf  Comicila 
Flight  Hours  in  this  Aircraft 

Flight  Hours  in  this  Position  ^ _  ■ 

r— ■  -  ■  -1 

Aporoxiftiate  Aga  [  ' 

IL  CREW  EFFECTIVENESS  MARKERS 


I  HOURS  OaSEAVCD 

Csotain  First  Offictr 


I  legs  OBSERVED  j 


Right  Engmsar 


1. 

6n«fings  (conduct  »nd  quslitvl 

None/Poor 

1 

7 

2 

A 

5 

Ejicallent 

2. 

Inquiry/Auorbon/Advocacy 

None/Poor 

1 

2 

3 

4 

5 

Excellent 

3. 

0«vv  self-critiqug  Idictsioni  and  actional 

None.'Poor 

1 

2 

3 

4 

5 

Excellent 

4 

Communication  SiOaasicns 

Poor 

1 

2 

3 

4 

Excellent 

5 

Leadersnio-Pc;io»*arjhiO''Concem  for  taaiu 

Poor 

1 

2 

3 

4 

r 

Excellent 

6. 

Inieroarjonal  rglaDonsnips/GrouO  Climata 

Poor 

1 

2 

3 

A 

5 

Excellent 

7. 

PraoaraUon/Plinnino/Vioilanca 

Poor 

1 

2 

3 

4 

5 

E.xcaileni 

8. 

WoiVload  diatnbutao/Diatracooni  avoided 

Poor 

1 

2 

3 

4 

5 

Excellent 

9. 

Overall  TTCHMCAL  proficiancy 

Poor 

1 

2 

3 

4 

5 

Excellent 

10. 

Overall  CREW  iffectnraness 

Poor 

1 

2 

3 

4 

5 

Excellent 

SUPPLEMENTARY  INFORMATION: 

1  1 .  Environmernat  worKioad 

Low 

1 

2 

3 

4 

5 

High 

Other  condibons  which  iignificandy  Influancad  the  flight  {include  weather.  ATC.  pre-existirg  mecharicais, 
in*fhght  abnormal  events,  ICE,  extra  crewmembers,  etc.).  Descnbe  below. 


86 


IV.  SPECIAL  CIRCUMSTANCES;  This  ssctian  providas  dita  on  non-standard  situations  or 
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in  Section  II.  Also  comment  on  extrema  (1  or  5)  niongs  from  Section  II. 
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SELECTED  COMMENTS  FROM  THE  ADVANCED  CRM 
INSTRUCTORS 
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727 

1.  Felt  Captain  did  not  establish  open,  fluid  communications  with  "A"  line 
Flight  Attendant.  It  was  very  brief  with  no  opportunity  for  feedback  and 
accomplished  after  arrival  of  final  paperwork.  Captain  was  extremely 
prorii..:cnt,  just  so  quiet  it  seemed  to  hamper  good,  open  communications 
within  crew.  First  Officer  exhibited  "nonchalant"  attitude  toward  flight 
engineer.  It  seemed  a  little  disrespectful  on  occasion.  Second  officer  was 
working  hard  to  be  a  part  of  "loop"  and  challenged  incorrect  altitude  in 
altitude  selector  window  with  proper  level  of  assertive  statement.  Captain  did 
set  high  standards  and  expectations  for  safe,  efficient  flight. 

2.  Captain  never  communicated  his  plans.  I-eft  First  Officer  out  of  loop. 
First  Officer  never  made  attempt  to  draw  out  plans.  Captain  was  very  low 
on  approach,  RED/RED  probably  on  purpose,  but  nothing  was  said  nor  any 
questions  asked. 

3.  Captain  set  tone  by  introducing  First  Officer  and  Second  Officer  as  "best 
you'll  ever  seel"  That  invited  crew  to  enter  in.  First  Officer  asked  about 
peculiarities  which  might  exist  at  destination  he  had  not  previously  flown  to. 
He  took  initiative  to  use  engine  anti-ice  when  no  one  else  did;  Captain  had 
established  atmosphere  which  invited  inquiry  and  critique;  Climate  was 
exceptionally  amicable,  relaxed  and  at  the  same  time,  professional. 


4.  Captain  appeared  new  to  tlic  aircraft  and  while  he  didn't  self  disclose  this 
to  me  the  way  he  was  using  the  talents  of  a  seasoned  First  Officer  and 
Second  Officer  led  me  to  believe  tliis.  He  gave  one  of  the  most  tliorough 
relevant  briefings  I  nave  yet  seen^  and  the  crew  responded  to  this  as  ^  team 
and  they  would  work  for  and  with  this  Captain  to  accomplish  the  task  at 
hand.  Weather  was  a  factor  and  the  crew  did  well,  First  Officer  and  Second 
Officer  used  crewmember  diplomacy  to  help  guide  some  of  the  tasks  like 
should  we  seat  the  Flight  Attendants,  etc.,  during  f  igh  work  loads.  The 
abnormal  was  the  amber  LED  indicator  in  flight.  This  was  caused  by  airfoil 
anti-ice,  but  1  was  the  only  one  who  had  .seen  that  before.  The  crew  Second 
Officer  went  through  the  hook  and  saw  the  note  about  A/I  and  the  Second 
Officer  and  suggested  that  we  turn  it  off  to  see  if  that  was  all,  Captain  said 
okay  and  the  light  went  out.  They  discussed  if  A/1  was  necessary,  and 
decided  no  and  left  the  wing  A/I  off.  Good  job! 

5.  Captain  very  knowledgeable,  dominated  crew  and  hampered  input. 
Second  Officer  a[ipeare<l  removed  and  alienated. 

m 

6.  Quiet  cockpit  very  little  communication  or  coordination.  Captain  would 
change  switch  positions  without  communicating  to  First  Officer.  First 
Officer  would  later  be  surprised  to  find  switch  in  the  new  position.  Captain 
relied  completely  on  autopilot.  Captain  would  set  up  altitude  capture,  for 
example,  and  go  on  to  something  new  and  never  monitor  if  altitude  capture 
was  taking  place.  Solo  operation. 
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7.  This  was  a  very  uneventful  segment  flown  between  two  high  density 
airjxjrts.  Crew  interacted  very  effectively.  Though,  interestingly  the  one 
time  the  Captain  failed  to  communicate  his  plan  (a  descent  clearance)  he 
entered  the  data  in  the  Flight  Management  System  incorrectly  and  it  was 
several  minutes  before  the  First  Officer  discovered  the  error.  First  Officer 
asked  that  I  relay  his  dissatisfaction  uith  procedural  change  not  allowing  pilot 
flying  to  maneuver  his/her  own  flaps.  Captain  noted  that  new'  young  First 
Officers  are  extremely  well-trained  and  eager  to  participate.  He  went  further 
in  saying  that  if  there  is  any  one  group  which  is  difficult  to  work  with  it  is  the 
"academy  guys". 

757/767 

8.  Departure  was  delayed  due  to  late  aircraft  arrival.  Captain  was  on  top  of 
situation  and  kept  passengers  and  Flight  Attendants  informed  of  progress 
and  expected  departure  time.  Crew  worked  well  together  although  the 
relation  was  not  overly  friendly,  (large  age  spread):  Senior  Captain— Junior 
First  Officer).  During  last  minute  runway  change  they  helped  each  other  get 
the  magic  setup  without  heads  down  time.  Captain  supported  Flight 
Attendant  in  Charge  with  a  problem  with  one  of  the  Flight  Aitendants-good 
mutual  respect,  even  though  his  briefings  were  "short". 

767 

9.  Captain  very  overpowering  and  strong  willed.  Within  first  2  minutes 
told  me  he  wa.c  a  fichter  oilot  and  if  Delta  would  hire  a"  fightci  jdlots.  tiu  y 


would  not  ne«d  CRM  because  tlicy  wouldn't  have  any  problems.  Played 
tricks  on  First  Officer  Odte  changing  radio  frequency  to  nonworking 
frequency).  Overconfident,  never  opened  flight  kit,  flew  departure  and 
approach  without  Jepp's.  First  Officer  humbled  himself  for  survival 
purposes,  could  not  have  been  assertive,  adapted  well  to  Captain  for 
effective  teamwork. 

10.  Captain  irritated  because  of  a  problem  with  his  chief  pilot-could  not 
keep  his  mind  on  the  task. 

1 1 .  Captain  did  not  ask  or  communicate  anything  to  First  Officer  during  the 
leg.  'file  first  time  the  First  Officer  or  I  knew  something  was  going  to  happen 
is  wlicn  tlic  Captain  picked  up  Lhe  microphone  ?jid  asked  ATC  for  deviations, 
direct  routes,  climb  for  turn  or  early  descent-and  it  was  the  First  Officers  leg! 

12.  A  one-man  show,  no  resource  management.  All  the  requests  were 
reasonable  and  appropriate  for  the  situation  and  the  First  Officer  .seemed  to 
have  adapted  to  this  style  but  die  flow  of  info  was  zero.  Both  front  to  back 
(Captain  to  cabin)  and  left  to  right  (Captain  to  First  Officer). 

MD-88 

13.  During  ground  (taxi)  congestion,  Captain  became  annoyed  and 
"stepped"  on  both  his  co-pilot  and  several  other  aircraft  to  obtain  taxi 
instructions.  In  rny  own  opinion,  co-piloi  had  called  as  much  as  possible. 
Captain  ilicn  told  co-pilot  he  was  not  aggres.sive  enough  (since  he  had  finally 
ubtained  clearance).  Neither  crewmember  was  comfortable  in  critiquing 
cc‘i.jri3  and  reviewing  results  because  atmosphcic  was  not  comtortabie. 


Caplain  seamed  to  dominate  both  actions  and  words.  Both  crewmembers 
were  dedicated  to  getting  job  done,  just  done  in  very  "authoritative" 
atmosphere. 

14.  Both  Captain  and  First  Officer  were  excellent  about  sharing  information 
concerning  plans  for  action.  Descent,  somewhat  abnormal  due  to  very 
structured  profile,  was  well  briefed  and  altitudes  double  cross-checked  to 
insure  compliance.  Captain  was  quiet  by  nature,  but  could  have  imparted 
more  information  about  flight  to  Flight  Attendants  and  set  better  cock'pit 
atmosphcie.  When  I  introduced  myself,  neither  pilot  responded  with  their 
own  names,  but  turned  back  and  looked  forward  during  very'  low  workload 
time.  After  15-20  minutes  atmosphere  relaxed  some,  and  good  discussion 
ensued. 

15.  Captain  set  excellent  tone/atmosphere.  Allcw-ed  First  Officer  to 
challenge  and  questions  were  encouraged.  An  example  was  a  log  book 
write-u[)  made  by  Captain.  Captain  read  tlic  write-up  to  the  First  Officer 
and  asked  for  comments  and  any  other  observations.  First  Officer  was 
comfortable  asking  questions  and  quick  to  want  to  keep  in  the  loop  of  info 
flow. 

16.  Crew  did  exceptional  job  relating  delayed  departure  info  to  ager.u;  and 
flight  attendants.  Also  kept  passengers  informed  of  status.  During  high  work 
ioad  periods,  task  were  well  divided  and  prioritized.  One  communications 
area  seemed  weak.  .Assumed  Flight  Attendants  were  knowledgeable  of 
current  abnormal  situation  without  any  communications  taking  place.  One 


crew  member  was  reluctant  to  turn  around  and  address  Flight  Attendants 
directly  with  complete,  appropriate  feedback  to  ensure  communications  loop. 

17.  Captain  was  speaking  with  such  quiet,  incomplete  ./ay,  it  was  difficult 
to  understand  him.  He  directed  First  Officer  to  work  on  minor  problem  to 
the  point  normal  inflight  duties  suffered.  The  "plan*  or  actions  occurring 
were  not  communicated  well  by  Captain  or  First  Offeer,  probably  because 
of  preoccupation  with  minor  maintenance  problem.  First  Offeer  did  on 
occasion  question  Captain  on  plan  for  approach  and  try  and  keep  other  crew 
members  in  loop. 

18.  In  my  ten  years  of  ser\'ice,  this  review  is  of  one  of  the  best,  most 
effcctis'c,  fight  crews  I  have  seen.  The  Captain  led  tlic  way  with  a  dynamic 
personality  and  positive  can  do  attitude.  His  humor  and  hard  work  ethic  set 
an  extremely  good  tone.  The  other  crew  members  f  ourished  in  this 
atmosphere  working  hard  (as  hard  as  possible)  to  accomplish  Uie  task  and 
give  the  best  serN'ice  possible  to  "our"  customers.  Even  as  1  do  this  rating  a 
day  later,  and  not  being  a  part  ot  *be  original  crew,  I  can't  help  fee!  1  was 
an  integral  part  of  this  crew  on  this  particular  fight  segment.  The  First 
Offeer  added  immensely  by  his  technical  profcier.cy  and  "team  member" 
attitude. 

19.  I  was  with  this  crew  for  3  legs  plus  the  layover.  The  frst  leg  was  short 
with  Weather  and  about  as  high  a  workload  you  can  get  in  a  2-man 
operation.  The  Captain  was  too  busy  to  "play  the  gam.e  of  CRM"  so  all  tl'c 
positive  things  he  did  I  tliink  are  die  norms.  Very  good  job  by  both  of  the 


crew  both  from  the  operations  point  of  view  and  tlie  CRM  standpoint.  This 
i.s  the  way  1  wish  all  the  operation  was.  Good  climate  for  open 
communication  all  ways,  super  job. 
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20.  From  the  outset,  it  seemed  that  the  Captain  was  not  pleased  to  have  me 
in  the  cockpit.  He  made  several  negative  references  to  CRM  how'cver  this 
did  not  interfere  with  effective  crew  interaction.  Captain  briefings  w'ere 
very  thorough,  referencing  manuals  and  proceduml  plates.  He  did  not 
hesitate  to  acknowledge  confusion  and  obtain  clarification  over  a  procedure 
or  clearance  that  might  have  been  misunderstood.  The  climate  for  effective 
conmuiaicatio!,  was  exceptionally  good.  Subordinates  did  not  hesitate  to 
express  opinions  counter  to  Captain's,  and  all  such  interactions  was  done 
with  good  humor.  Second  Officer  w'as  particularly  adroit  at  accommodating 
both  Captain  and  First  Officer  w'hile  making  a  P.\  concerning  flight 
progress. 

UNIOFNIIFIRn 

21.  Captain  used  Flight  Management  System  at  1300'  AGL  on  climbout 
instead  of  monitoring  inslruments/traffic.  Pilot  Flying  (First  Officer)  copied 
AT!S.  Did  not  ask  Pilot  Not  Flying  to  act  as  teammate.  Captain  took 
aircraft  from  Pilot  Flying  (First  Officer)  just  after  touchdown  at  high  speed. 

22.  This  crew  had  minimum  communication  for  entire  flight.  Captain  was 
not  in  aircraft  until  3  minutes  prior  to  scheduled  a  pushback.  Before  start 
checklist  road  after  papers  were  received  and  doors  closed.  No  talking  in 


cruise  and  Captain  tunetl  his  own  navigation  radios  for  approach  even 
though  he  was  flying. 

23.  An  excellent  crew  that  relaxed  and  worked  well  as  a  team.  Second 
Officer  felt  comfortable  enough  to  point  out  switches  in  wrong  position. 

First  Officer  flew  leg  and  made  approach  to  near  minimums.  Planned  aJiead 
and  did  very  well,  Captain  was  excellent  Pilot  Not  Flying  and  everyone 
knew  their  role. 

24.  On  a  short  flight  leg  this  aircraft  W'as  held  up  extremely  high  because  of 
lower  traffic.  First  Officer  fust  staricd  expressing  concern  by  making 
several  references  to  altitude  in  relationship  to  distance  from  the  airport. 
Finally  made  an  excellent  series  of  inquiries;  "Do  you  want  to  S-turn?", 

"Do  you  want  a  360  degree  turn?"  Through  tlie  passage  of  timely  and 
accurate  information,  successful  inquiries,  and  coordinate  total  crew  effort 
the  landing  was  safe  and  uneventful.  Gr-'  'o  v/atch!! 

25.  Good  crew.  Captain  was  very  active  .  decision  making,  planning  and 
communicating.  Also  very  active  in  actions,  sometimes  unfortunately  so. 

On  First  Officer's  leg  Captain  was  still  on  controls  at  80  KIAS  on  takeoff 
roll,  confusing  First  Officer  as  to  who  was  flying.  No  positive  change  of 
control  then  or  on  landing  roll.  Captain  also  made  autopilot  inputs  in  flight 
while  First  Officer  was  Pilot  Flying  and  didn't  tell  him  about  it.  Overall 
though,  a  very  good  crew. 


26.  Co-pilot  was  Pilot  Flying.  He  did  not  ask  tlie  Captain  to  do  any  Pilot 
Not  Flying  duties  and  Captain  didn't  volunteer  to  do  them.  Pilot  Flying  did 
most  everything  by  himself. 

27.  This  crew  was  dominated  by  two  young  "fighter  ty^rcs".  Tney  were  not 
disrespectful,  but  neither  were  they  strong  team  players.  Captain  maintained 
deconim  but  the  atmosphere  was  icy  and  therefore  lacked  the  sort  of 
interaction  that  would  include  thorough  briefings  and  discussion  of  possible 
irregularities. 

28.  Captain  gave  most  effective  Flight  Attendant  briefing  1  have  ever  heard. 
Both  First  Officer  and  Second  Officer  practiced  effective  assertion.  First 
Officer  was  female  and  very  disturbed  at  treatment  she  had  just  received  by 
chief  pilot  over  a  problem  with  a  Captain.  She  was  angry  and  bitter  and 
rightfully  so.  She  did  not  allow  this  to  affect  her  professionalism.  Second 
Officer  was  an  older  man  and  had  been  with  five  airlines.  He  seemed  to  be 
competing  with  Capuiin  and  somewhat  argumentative.  Howc'.’cr,  Captain 
was  still  able  to  build  an  effective  team. 
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